1. Field of the Invention
The present invention relates generally to digital imaging. More particularly, the present invention relates to a method and device applying digital image filtering to CFA data.
2. Description of the Related Art
In a typical digital imaging system, a digital camera captures and stores an image into a video buffer, e.g., in RGB or other multi-plane format. The data is then fetched from the video buffer and sent to a digital display, such as an LCD panel.
Typically, small battery-powered digital imaging devices such as video or still cameras, and devices incorporating such cameras, such as cell phones, personal digital assistants (PDAs), etc., have not been considered to be powerful enough to provide digital image processing capability. Specifically, such devices lack the processor and memory requirements to add digital filter effects such as soften, blur, emboss, directional blur, sepia, or black & white (grey scale) effects.
In prior art imaging systems, the user would compose the picture and commit it to memory prior to selecting or applying a digital filter effect to the image. For example, in the ordinary sequence of events of taking a picture using a digital camera, a photographer would first compose a picture, for example, by aiming a camera at a subject. When the photographer is happy with the composition, he or she “takes the picture,” by storing the image in memory. After the image is in local memory, the photographer may then select a filter effect to apply to the image and apply the effect. Typically, a filter effect will be applied after the image is downloaded to a personal computer having increased memory and processing power. After applying the filter effect, the modified image is displayed for the photographer to view. If the photographer likes the resulting filtered image, he or she may keep the image, e.g., by printing it or storing it in non-volatile memory. In the case of camera-enabled communication devices, such as cell phones, the user may transmit the image to a remote location. However, if the photographer decides the resulting image is not desired, he or she may undo the filter operation, and either decide to apply a different filter, or no filter at all.
The preceding description is very acceptable for high powered machines such as personal computers, to execute the software filter reasonably fast. However, for many imaging devices such as small digital cameras, cell phones, and other devices, the memory and processing requirements are excessive.
For example, a camera having a 3.8 Mega-pixel sensor has an image resolution of 2272×1704 pixels. This means that the total number of pixels representing the image is 3,871,488 pixels. Each pixel may consist of 8 bits providing the red component, 8 bits for the green component, and 8 bits for the blue component, for a total of 3 bytes or 24 bits per pixel.
Many filter effects are applied by the well-known convolution filter. Such filter effects include soften, blur, edge detection, emboss, etc. For a simple implementation, assume the digital filter circuitry fetches 9 pixels in a 3×3 matrix 80 as shown in FIG. 1, and performs a particular function depending upon the filter be applied. Each matrix includes 9 pixels labeled P1 to P9 as shown in FIG. 1 with pixel P5 in the center. Pixel P5 may be substituted with a new pixel P5′ based on the neighboring pixel values, e.g., by the formula:
                              P          5          ′                =                              (                                                            P                  1                                ⁢                                  C                  1                                            +                                                P                  2                                ⁢                                  C                  2                                            +                                                P                  3                                ⁢                                  C                  3                                            +                                                P                  4                                ⁢                                  C                  4                                            +                                                P                  5                                ⁢                                  C                  5                                            +                                                P                  6                                ⁢                                  C                  6                                            +                                                P                  7                                ⁢                                  C                  7                                            +                                                P                  8                                ⁢                                  C                  8                                            +                                                P                  9                                ⁢                                  C                  9                                                      )                    /          Div                                    (        1        )            wherein C1 to C9 are constants and Div is a divide value. Each pixel P is defined by red, green, and blue intensity values. For pixels defined by 24 bits per pixel, typically each red, green, and blue intensity value will be represented by a number ranging from 0 to 255. Thus, for each pixel, Equation 1 is calculated three times, i.e., once for each color value. In addition each color component for P5′ is constrained to a minimum of zero and a maximum of 255. Various effects can be achieved depending on the constant values chosen. Common convolution filter effects include blur and soften effects. Other possible effects include sharpen, edge detection, emboss, and directional blur. Convolution filter effects are well known and understood in the image-processing field.
Under this method, the total bytes transferred to the camera interface from the camera is three times the number of pixels (at 3 bytes per pixel) or 11.06 M bytes (where 1 M byte=1024 K bytes, and 1 K byte=1024 bytes). Secondly, the number of accesses required to memory is 9 reads times the number of pixels, which is 3,866,944+1 write times the number of pixels. In the example, this works out to 38,669,440 memory accesses for the exemplary filter implementation. This is a large processing task for small battery-powered devices.
It has been known to provide a real-time digital image preview via a digital display that users can use when composing a shot. Many users prefer to see such a live, real-time, digital image preview rather than looking through a view-finder to frame and compose the image. However, such real-time previews have not included image enhancement or other processing or filtering.
The problem of providing the enhanced functionality of image processing to an imaging device without large memory and processing requirements has not been adequately addressed prior to the present invention.